The aluminum (Al) cation Al3+ is highly rhizotoxic and is a major stress factor to plants on acid soils, which cover large areas of tropical and boreal regions. Many woody plant species are native to acid soils and are well adapted to high Al3+ conditions. In tropical regions, both woody Al accumulator and non-Al accumulator plants occur, whereas in boreal regions woody plants are non-Al accumulators. The mechanisms of these adaptations can be divided into those that facilitate the exclusion of Al3+ from root cells (exclusion mechanisms) and those that enable plants to tolerate Al3+ once it has entered the root and shoot symplast (internal tolerance mechanisms). The biochemical and molecular basis of these mechanisms have been intensively studied in several crop plants and the model plant Arabidopsis. In this review, we examine the current understanding of Al3+ exclusion and tolerance mechanisms from woody plants. In addition, we discuss the ecology of woody non-Al accumulator and Al accumulator plants, and present examples of Al3+ adaptations in woody plant populations. This paper complements previous reviews focusing on crop plants and provides insights into evolutionary processes operating in plant communities that are widespread on acid soils.